Novel Bounds on the Capacity of the 
Binary Deletion Channel 



Dario Fertonani and Tolga M. Duman 

Department of Electrical Engineering 
Arizona State University 
Tempe, AZ 85287-5706 
E-Mail: dario.fertonani@asu.edu, duman@asu.edu 

Submission: October 4, 2008 



Abstract 

We present novel bounds on the capacity of the independent and identically distributed binary 
deletion channel. Four upper bounds are obtained by providing the transmitter and the receiver with 
genie-aided information on suitably-defined random processes. Since some of the proposed bounds 
involve infinite series, we also introduce provable inequalities that lead to more manageable results. For 
most values of the deletion probability, these bounds improve the existing ones and significantly narrow 
the gap with the available lower bounds. Exploiting the same auxiliary processes, we also derive, as a 
by-product, a couple of very simple lower bounds on the channel capacity, which, for low values of the 
deletion probability, are almost as good as the best existing lower bounds. 

Index Terms 

Binary deletion channel, channel capacity, capacity bounds. 

I. Introduction 

We consider a binary deletion channel where each bit in the input sequence gets deleted, 
independently of the others, with probability d, while the non-deleted bits are received without 
errors and in the correct order. The positions at which the deletions occur are unknown to both 
the transmitter and the receiver. Formally, let X = {Xn}n=i be a sequence of N bits at the input 
of the channel, let M be the number of received bits, which is a random variable taking values 
in {0, 1, ... , A^} according to the realization of the deletion process, and let Y = {y„}^l^ be the 
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received sequence. The capacity per input bit of this channel, generally referred to as independent 
and identically distributed (IID) binary deletion channel, is defined as [1] 

C = lim max— /(X: Y) (1) 

TV^oo P(X) ^ ' ^ ^ 

where P(X) is the distribution of the input sequence, and /(■; ■) is the average mutual information 
between two random sequences [2] . The capacity ([T]) is unknown, and only some upper and lower 
bounds are available in the current literature. 

The first lower bound on the capacity of the deletion channel was derived by Gallager in [3], 
where he proved that, for d < 0.5, the capacity of interest is at least equal to that of a binary 
symmetric channel with bit-flipping probability d. A number of lower bounds have since been 
proposed (see [4], [5], and references therein), among which the best bounds that we are aware 
of are the ones presented in [4] and [5]. In particular, the latter bound outperforms the former 
when d < 0.35, that is, for all values of d for which the authors of [5] could run the required 
computations whose execution time grows quickly as d increases. Throughout the paper, the 
reference lower bound will thus be the one in [5] for d < 0.35 and the one in [4] for d > 0.35. 

Only a few upper bounds have been derived on the capacity of the IID deletion channel. A 
simple upper bound is given by the capacity of an IID erasure channel with erasure probability d, 
since the erasure channel is identical to the deletion channel, except that the receiver additionally 
knows the positions of the deleted bits [2]. A combinatorial bound proposed by UUman in [6], 
which was originally derived for particular channels with synchronization errors, had been used 
for decades as an upper bound for the deletion channel. However, it is not a true upper bound, 
and it has been recently found to be violated by provable lower bounds on the capacity of 
the deletion channel [4]. The reason is due to the fact that UUman focused on systems with 
null error probability, while the definition of capacity relies on the weaker condition of error 
probability that can be made arbitrarily low by increasing the length of the codewords [2]. The 
only non-trivial upper bound that we are aware of is the one presented in [7], which will be 
adopted here as a reference benchmark. 

This paper presents novel upper bounds on the capacity of the IID deletion channel that 
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improve the existing ones for most values of the deletion probability d. All upper bounds are 
computed by considering the capacity of some auxiliary channels obtained by providing genie- 
aided information on suitable random processes related to the deletion process. In particular, 
we show that, when such auxiliary random processes are revealed to the transmitter and/or the 
receiver, we obtain memoryless channels whose capacity can be evaluated by means of the 
Blahut-Arimoto algorithm (BAA) [8], [9], leading to provable upper bounds on the capacity of 
interest. Moreover, we show that, based on the introduced auxiliary processes, lower bounds on 
the capacity of the deletion channel can be derived as well. The obtained lower bounds, yet 
close to the ones proposed in [4] and [5] for low values of d, do not improve them, and will 
only be considered as by-product results. 

The paper is organized as follows. Section |Il] introduces an auxiliary channel based on which 
we derive three upper bounds on the capacity of the IID deletion channel, which are presented in 
Sections Unl JVl and |Vl respectively. The fourth upper bound, evaluated by exploiting a different 
auxiliary channel, is introduced in Section |VIl The main contributions in upper bounding the 
capacity of the deletion channel are summarized an discussed in Section IVIIl Finally, Section IVIIIl 
introduces a couple of simple lower bounds, while Section |IX] gives some concluding remarks. 

II. A Useful Auxiliary Channel 

Let L and R be two natural numbers such that R < L, and let us define D = L — R. 
We consider a channel for which, at each use, the input consists of a sequence of L bits and 
the output consists of a sequence of R bits. The input/output relationship characterizing each 
channel use is the following: D bits are deleted from the L input bits, while the remaining 
R bits are received without errors and in the correct order. At each channel use, the deletion 
pattern, that is, the positions at which the D deletions occur, randomly takes on each of the 
possible (^) realizations with equal probability, and is unknown to both the transmitter and the 
receiver. Also, deletion patterns in different channel uses are independent, so that the channel 
is memoryless. As an example, the transition probabilities characterizing the use of the channel 
are reported in Table ID for the case L = 3 and R = 2. A and B denote the input sequence and 
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TABLE I 



Transition probabilities for the auxiliary channel. 



the output sequence, respectively, while P{-\-) denotes conditional probability. 
The capacity per use of the considered auxiliary channel is defined as 

/(L,i?) = max /(A; B), (2) 

where P(A) is the distribution of the input sequence. Since each channel output is a sequence 
of R bits, the following upper bound holds 

fiL,R)<R. (3) 

In some particular cases, it can be shown that f(L, R) achieves the upper bound ([3]). These cases 
are listed and briefly discussed in the following. 

• /(L, 0) = 0. All input bits are deleted and no information can be delivered. 

• /(L, 1) = 1. A capacity-achieving scheme consists of transmitting, at each channel use, 
either a sequence of L zeros or a sequence of L ones, with equal probability and in- 
dependently of the previous/future transmissions. In this case, for each channel use, the 
only received bit fully determines the input sequence, irrespectively of the deletion pattern. 
Formally, adopting the standard notation for the entropy and the conditional entropy [2], 
we get 

/(A; B) = H{A) - H{A\B) = H{A) = 1 
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TABLE II 
Capacity /(L, ii). 



which achieves the upper bound Q. 
• f{L, L) = L. Since all transmitted bits are correctly received, the capacity is equal to L bits 
per channel use, which is achieved by independent and uniformly distributed (lUD) input 
bits. 

When R ^ {0, 1, L}, we could not find a closed-form expression of the capacity f{L, R). On the 
other hand, since the auxiliary channel is memoryless and has finite input/output alphabets, its 
capacity can be numerically evaluated by means of the BAA [8], [9]. To run the BAA, we only 
need the transition probabilities characterizing the channel, as those reported in Table HI Hence, 
in principle, we can compute the capacity f{L, R) based on similar tables, for all desired values 
of L and R. Unfortunately, the implementation of the BAA becomes computationally infeasible 
for large values of L — for example, L = 17 is the largest value that we were able to manage 
for all possible values of R, while L = 22 is the largest value that we were able to manage 
for R = L — 1, which will be shown later to be a case of particular interest. Some values 
of f{L,R) are reported in Table [III where the results obtained by means of the BAA have a 
two-digit precision after the decimal point, and are rounded up to the next hundredth since, 
rigorously, the BAA can underestimate the true capacity if a finite number of iterations are 
performed [8], [9]. 

In the following, we introduce several lemmas that will be used in the remaining sections 
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to manipulate the capacity of the auxiliary channel when running the BAA seems impossible. 
Before providing the lemmas, we define 

fiL,D) = fiL,L-D), (4) 

so that we can index the capacity of the auxiliary channel either by the number of received bits, 
using /(-, ■), or by the number of deleted bits, using /(-, ■). The following definitions will also 
be useful in the remaining sections: 

a{L,R) = R-f{L,R), (5) 
a{L,D) = a{L,L-D) = L-D-f{L,D). (6) 

Note that the coefficients a{-,-) and a{-,-) cannot be negative due to Q. 
Lemma 1: For all values of L and R, the following holds 

/(L + l,i?) </(L,i?) . (7) 

The proof is based on the fact that, when additional information is provided to the transmitter, the 
capacity of a system cannot decrease [2]. In particular, the capacity f{L+ 1, R) cannot decrease 
if, at each channel use, the transmitter knows one of the positions at which the L + l — R deletions 
occur. Clearly, the bit transmitted in that position is irrelevant. Moreover, if the revealed position 
is chosen according to a uniform distribution on the L + l — R possible values, the system is 
characterized by L effective input bits, R output bits, and lUD deletion patterns, that is, by 
definition, a system with capacity f{L,R). Hence, the lemma is proved. 
Lemma 2: For all values of L and R, the following holds 

if L>L then a{L, R) > a{L, R) . (8) 

The proof that a{L + 1, R) > a{L, R) is simply derived from ([5]) and ©. The remainder of the 
lemma can then be proved by induction. 

Lemma 3: For all values of L and all positive values of D, the following holds 

f(L + l,D)<fiL,D-l)j^ + 



fiL,D) + l 



1 - 



(9) 
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The proof is based on the fact that, when additional information is provided to both the transmitter 
and the receiver, the capacity of a system cannot decrease [2]. In particular, we consider the 
information on the binary event "the last bit of the L + 1 transmitted bits is deleted", which occurs 
with probability D/{L + 1). When the event occurs, the last transmitted bit is irrelevant and the 
system is characterized by L effective input bits and D — 1 deletions on lUD positions, that is, 
the system has capacity f(L,D — l). When the event does not occur, the last transmitted bit can 
be safely sent uncoded, while, for the first L transmitted bits, the systems is characterized by 
L effective input bits and D deletions on lUD positions, that is, the system has capacity f{L, D). 
Hence, the lemma is proved. 

Lemma 4: For all values of L and all values of D, the following holds 

a{L +l,D)> &iL, D) - . (10) 

The lemma is proved after straightforward manipulations of dH) based on ([3]) and 

The lemmas provided hereafter focus on a particular case, that is, the occurrence of exactly 
one deletion. The reader may skip them without affecting the arguments exploited in Sections Unl 
HVl rvl and |VIl The interest for this case will become evident in Section IVIII 
Lemma 5: For all values of L, the following holds 

/(nL,l)</(L,l) + (n-l)L, Vn > . (11) 

Let us partition the input sequence of nL bits into n subsequences of L consecutive bits, and 
let us assume that both the transmitter and the receiver knows in which of the subsequences 
the deletion occurs. By definition, this subsequence has capacity /(L, 1), while the remaining 
n — 1 subsequences have capacity L. Hence, since the capacity f{nL, 1) cannot exceed that of 
the described genie-aided system, the lemma is proved. 
Lemma 6: For all values of L, the following holds 

a(nL, 1) > a(L,l), Vn > . (12) 

The lemma directly follows from (fTTl) by definition Q. 
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Lemma 7: For all values of L, the following holds 

f{L + 1, 1) > /(L, 1) + 1 - - h (13) 

where h{-) is the binary entropy function [2]. 

To prove the lemma, we first notice that the equation 

/(A; B) = /(A; B, C) - /(A; C|B) 

holds irrespectively of the definition of the random processes A, B, and C [2]. Moreover, since 
/(A; C|B) cannot be larger than the entropy H{C) of the process C, we can write 

/(A;B) > /(A;B,C) -i/(C) . (14) 

In particular, let A and B be, respectively, the input sequence and the output sequence of the 
auxiliary channel considered in this section, when the input sequence includes L + 1 bits and 
exactly one deletion occurs. Also, let C be the binary event "the last bit of the L + 1 transmitted 
bits is deleted", whose entropy is H(C) = h{l/{L + 1)). Under these definitions, the inequality 

/(L + 1, 1) > max /(A; B, C) - /i {-^\ (15) 
P(A) \L+lJ 

follows from (fT4l) . Note that the first term at the right-hand side of (fT5l) is the capacity of a 

channel identical to the considered one, when the receiver is provided with side information on 

the event C, while the transmitter is not. According to the data-processing inequality [2], the 

capacity of this genie-aided system does not increase if, when the event C occurs, the receiver 

deletes one of the received bits, selected with equal probability over the L received bits. In 

this case, the channel consists of two independent subchannels: the former is characterized by 

L input bits and one deletion on lUD positions, and thus has capacity /(L, 1), while the latter 

is an erasure channel with erasure probability 1/(L + 1), and thus has capacity 1 — + 1). 

Hence, we can write 

max /(A; B, C) > /(L, 1) + 1 ^— 

which, combined with (fT5l) . proves the lemma. 
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Lemma 8: The following holds 



,„„ . 1 . (16) 

To prove the lemma, we first notice that the inequality 

a(L + 1, 1) < a{L, 1) + + h (;^) (17) 

directly follows from (fT3l) by definition Then, according to (flOl) and (flTl) . we can write 

1 + 1) 1 1/1 



L + 1- a(L, 1) - 6(L, 1)(L + 1) a(L, 1) + 1 
which proves the lemma since both sides tend to one as L tends to infinity. 
Lemma 9: The following holds 



lim 

L^oo L 



1 . (18) 



/(L + 1,1)-/(L,1 
To prove the lemma, we first notice that the inequalities 

^ - m - " (m) ^ [f^' ^ ^- ^) - f^'- ^)] ^ ^ ^ '-^r^ 

follow from ^ and (fT3l) after simple manipulations. The left-hand side in (fT9l) clearly tends 
to one as L tends to infinity. Then, we notice that the limit 

lim /<M = 1 

follows from the fact that the binary channel with one deletion tends to the binary identity 
channel, whose capacity per input bit is one, as the length of the input sequence tends to 
infinity. Hence, the right-hand side in (fT9l) tends to one as L tends to infinity, and the lemma is 
proved. Note that (fTSi) implies (fT6l) . but is stronger. 

in. The First Upper Bound 

In this section, we derive an upper bound on C by providing side information on a random 
process Z, defined in the following. Let D be a non-negative integer parameter and let us assume 
that the total number of deleted bits is a multiple of D + 1, so that S = {N - M)/{D + 1) is an 
integer — this assumption does not affect the capacity evaluation, where the limit — oo is to 
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^2 =4 





Fig. 1. A possible realization of the process Z, when D = 1 and 5 = 3. Each white square represents a transmitted bit that 
is correctly received, while each gray square represents a transmitted bit that is deleted. The positions of the bold-faced bits 
define the process Z. 



be considered. We define Z = {Zi]f^^ sucii tliat Zi is equal to the position in the transmitted 
sequence of the \D + l]-th deleted bit and, for each value of i in {2, 3, ... , S}, Zi is equal to 
the difference between the position in the transmitted sequence of the \{D + l)z]-th deleted bit 
and that of the \{D + — l)]-th deleted bit. An example is depicted in Fig. [Hand discussed in 
the related caption. Given the assumption of IID deletions, the process Z is IID too, and each 
element of Z takes on the value L + 1 with probability 

P{Z, = L + l)= d^+\l - d)^-^ (20) 



according to the Pascal distribution [10], for all values of L such that L > D.To point out various 
similarities between the bounds presented in this paper, it is useful to define, for L > R > 
and L > D > 0, the. terms 

p{L,R) = f^^ci^-«(l-rf)« (21) 



P{L,D) = p{L,L-D)=(^^y^{l-d)'^-^ (22) 

so that we get 

P{Z, = L + 1) = d-p{L,D) . (23) 

The realizations of the process Z are actually unknown to both the transmitter and the receiver. 
Hence, an upper bound on the capacity of the deletion channel can be obtained by providing 
them with genie-aided information on Z. We will refer to the capacity per input bit of this 
genie-aided system as Ci. With this side information, we have S blocks that do not interfere 
with each other, where the i-th block has Zi input bits, D + 1 of which get deleted. The last 



October 4, 2008 



DRAFT 



FERTONANIAND DUMAN: NOVEL BOUNDS ON THE CAPACITY OF THE BINARY DELETION CHANNEL II 

input bit of each block is irrelevant, since both the transmitter and the receiver know that it 
gets deleted. The z-th block is thus characterized by — 1 effective input bits and D deletions 
on lUD positions, so that the related capacity is f{Zi — 1, D), as defined in Section [III Hence, 
defining the expectation operator E[] and considering that 

N 

lim - = E[Z,] 

TV— >oo O 

by the law of large numbers [10], we get 

s 



^1 = 

N^oo JM ^ — ^ 
i=l 



E[Zi] 5-.00 S .^^ 

1 



fiZ,-l,D) 



where the last equality follows from the law of large numbers. Finally, by exploiting the properties 
of the Pascal distribution [10], the upper bound yields 

L=D 

which can be also written as 

L=D L=D 



l-d 



72 °° 



= l-d-^—Y,^{L,D)p{L,D). (24) 

L=D 

Since the coefficients «(■,■) cannot be negative, the bound (|24|) is at least as good as the trivial 
bound \ — d.\n particular, by combining Lemma 4 with the available outcomes of the BAA, it 
can be proved that the bound (l24l) equals \ — d when D = 0, otherwise it is strictly better. 

Unless D = 0, it seems infeasible to evaluate the coefficients D) for all values of L 
required in (l24l) . Let us assume that we know the coefficients q;(L, D) for all values of L such 
that L < Lmax, but not for larger values of L — in particular, we have Lmax = 17. In this case, 
we can exploit the inequality in (fTOl) to manipulate the coefficients a{L, D) for L > Lmax- The 

October 4, 2008 DRAFT 



FERTONANl AND DUMAN: NOVEL BOUNDS ON THE CAPACITY OF THE BINARY DELETION CHANNEL 



12 




obtained results are reported in Fig. [2l for all values ofDin{0,l,...,17} and Lmax = 17. The 
resulting bounds, referred to as CI, are actually larger than the capacity Ci in (|24)) . because of 
the use of (flOl) for L > Lmax- Hence, the reported curves can be improved when an inequality 
tighter than (flOl) is exploited to manipulate the coefficients a(L, D) for L > Lmax- In Fig. [21 
the upper bound proposed in [7] and the lower bounds proposed in [4] and [5], which are the 
best existing bounds that we are aware of, are also reported for comparison 3 We point out that 
the upper bound Cj* improves the upper bound presented in [7] for a wide range of d values, in 
particular when d > 0.35. 

IV. The Second Upper Bound 

In this section, we derive an upper bound on C by providing side information on a random 
process W, defined in the following. Let i? be a non-negative integer parameter and let us 
assume that the number of bits at the output of the deletion channel is a multiple of 7?+ 1, so that 

'As explained in Section U the lower bound proposed in [5] is adopted when d < 0.35, while the one proposed in [4] is 
adopted when d > 0.35. 
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Fig. 3. A possible realization of the process W, when 7? = 1 and T = 3. Each white square represents a transmitted bit that 
is correctly received, while each gray square represents a transmitted bit that is deleted. The positions of the bold-faced bits 
define the process W. 



T = M/ + is an integer — this assumption does not affect the capacity evaluation, as in the 
previous case. We define W = {PVj}^^ such that Wi is equal to the position in the transmitted 
sequence of the [R + l]-th received bit and, for each value of i in {2, 3, ... , T}, Wi is equal to 
the difference between the position in the transmitted sequence of the [{R + l)i]-th received bit 
and that of the \{R + — l)]-th received bit. An example is depicted in Fig. [3] and discussed 
in the related caption. Given the assumption of IID deletions, the process W is IID too, and 
each element of W takes on the value L + 1 with probability 

P{Wi = L + 1) = (1 - d)p{L, R) (25) 

according to the Pascal distribution [10], for all values of L such that L > R. 

As in the previous case, an upper bound on the capacity of the deletion channel can be obtained 
by providing the transmitter and the receiver with genie-aided information on the realizations 
of W. We will refer to the capacity per input bit of this genie-aided system as C2. Similarly to 
the previous case, we have T blocks that do not interfere with each other, the i-th block having 
Wi input bits and _R+ 1 output bits. The last input bit of each block can be safely sent uncoded, 
since both the transmitter and the receiver know that it is correctly received. Hence, following 
the same arguments as in the previous section, we get 



1=1 

T 



E[Wi] T^oo T f_ 

rE[f{W,-l,R) + l] . 
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Finally, by exploiting (1251) and the properties of the Pascal distribution, the upper bound yields 

L=R 

which can be also written as 

C2 = ^^^E(^+i)p(^'^)-^:R^E[^-/(^'^)b(^'^) 



L=B. L=R 



l-d 



= 1-^/-^— (26) 

+ L=R 

Since the coefficients ■) cannot be negative, the bound ([261) is at least as good as the trivial 
bound 1 — d. In particular, by combining Lemma 2 with the available outcomes of the BAA, it 
can be proved that the bound (l26l) equals 1 — d when R E {0, 1}, otherwise it is strictly better. 

When i? > 1, it seems infeasible to evaluate the coefficients a{L, R) for all values of L 
required in (l26l) . Let us assume that we know the coefficients a(L, R) for all values of L such 
that L < Lmax, but not for larger values of L — in particular, we have Lmax = 17. In this case, 
we can exploit ^ to manipulate the coefficients a(L, R) for L > Lmax, obtaining 

C; = ^--^ Yl H^MAx, R) - a{L, R)] p(L, i?) + (1 - d) 

+ L=R 



1 - 



(27) 



R+1 

after a few straightforward manipulations — the bound is referred to as C| because it is actually 
larger than the capacity C2 in (|26l) . The obtained results are reported in Fig. HI for all values 
of i? in {0, 1, ... , 17} and Lmax = 17. Clearly, such curves can be improved when an inequality 
tighter than dSl) is exploited to manipulate the coefficients a{L, R) for L > Lmax- In Fig- SI the 
upper bound proposed in [7] and the lower bounds proposed in [4] and [5] are also reported for 
comparison. We point out that the upper bound C2 improves the upper bound presented in [7] 
for most values of d, in particular when d > 0.1, and, for large values of d, the gap from the 
best lower bound is now roughly halved. 

V. The Third Upper Bound 

In this section, we derive an upper bound on C by providing side information on a random 
process V, defined in the following. Let L be a positive integer parameter, based on which 
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C2, from fi=0tofi=17 
Upper bound from [7] 
Lower bound from [4] and [5] 




0.0 0.1 



0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 
d 



Fig. 4. Different bounds on tiie capacity of the deletion channel. 



we partition the input sequence X into subsequences {Xj} of L consecutive bits. Formally, we 
define 

Xj = (X(j_i)2,+i, X(j__i)2,+2, . . . , XiL)^ Vi > 1 . 

For example, when L = 3, we have Xi = (Xi, X2, X3), X2 = (X4, X5, Xq), X3 = (X7, Xg, Xg), 
and so on. We assume that X is a multiple of L, and thus that there are exactly Q = N/L 
subsequences {Xj}^]^ — this assumption does not affect the capacity evaluation, as in the 
previous cases. We then partition the output sequence Y into Q subsequences {Y, }^^, where, for 
each value of i in {1, 2, ... , Q}, Yj includes the received bits related to the input subsequence Xj. 
Finally, we define the random process V = such that, for each value of ? in {1, 2, ... , Q}, 

Vi denotes the number of bits in the subsequence Yj. An example is depicted in Fig. [5] and 
discussed in the related caption. Given the assumption of IID deletions, the process V is IID 
too, and each element of V takes on the value R in {0, 1, . . . ,L} with probability p{L,R), 
according to the binomial distribution. 

As in the previous cases, an upper bound on the capacity of the deletion channel can be 
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Vi = 2 


V2 = 1 




V^4 = 

^> 



Fig. 5. A possible realization of tlie process V, wiien L = 3 and (5 = 4. Eacii whiite square represents a transmitted bit that 
is correctly received, while each gray square represents a transmitted bit that is deleted. 

obtained by providing the transmitter and the receiver with genie-aided information on the 
realizations of V. We will refer to the capacity per input bit of this genie-aided system as C3. 
Similarly to the previous cases, we have Q blocks that do not interfere with each other, the z-th 
block having L input bits and Vi output bits. Hence, using similar arguments as in the previous 
sections, we get 

1 

j=i 

1 ^ 

R=0 

which can be also written as 

L L 

R=0 R=0 

^ v ' 

1-d 

1 ^ 

= 1-d- -J2(^iL,R)p{L,R) . (28) 

R=0 

Hence, since the coefficients a{-,-) cannot be negative, the bound (l28l) is at least as good as the 
trivial bound 1 — rf. In particular, by combining Lemma 2 with the available outcomes of the 
BAA, it can be proved that the bound (l28l) equals 1 — d when L E {1,2}, otherwise it is strictly 
better. Note that, unlike the previous cases, the bound C3 does not involve an infinite series. 

The upper bound (|28] ) is plotted in Fig. [6l together with the upper bound proposed in [7] 
and the lower bounds proposed in [4] and [5]. For each value of L for which we could run the 
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BAA, the bound C3 improves as L increases — we conjecture that this behavior holds for any 
value of L (see Section IVIIl) . Note that the considered approach significantly improves the bound 
presented in [7] for most values of the deletion probability d, in particular when d > 0.08. 

VI. The Fourth Upper Bound 

Given any positive value of the integer parameter L, we can define a system identical to the 
deletion channel, in which the receiver knows the realizations of the process V defined in the 
previous section, while the transmitter does not. In this case, it is useful to think of the system 
as if there were a "parallel" channel that provides the sequence V to the receiver. The capacity 
per input bit of this system, which will be denoted by C4, is definitely an upper bound on the 
capacity ([T]), since, when the parallel output V is neglected, the original deletion channel is 
obtained. Moreover, the upper bound C4 cannot be larger than C3 for the same value of L, 
since the system with capacity C3 reduces to the system with capacity C4 when the transmitter 
neglects the side information on the process V. 

As for the system considered in the previous section, we have Q blocks that do not interfere 
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11 
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2d(l -d) 
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TABLE III 



Transition probabilities for the evaluation of d (L = 2). 



with each other, so that a discrete memory less channel results. For each use of this channel, 
we still have an input sequence of L bits and, with probability p(L,R), an output sequence of 
R bits, but now the value of R is unknown to the transmitter. Hence, all transmitted sequences 
must be taken from the same distribution, and no longer from a distribution matched to the 
number of deletions in the current channel use. Consequently, the results related to the auxiliary 
channel introduced in Section HI] cannot be exploited here. Formally, we get 

C4 = lim max ^/(X; Y, V) 

= - lim max ^/(X; Y, V) 

L Q^oo p(x) Q 

= i max /(X,; Yi) . (29) 
L P(x,) 

When L = 1, this auxiliary channel reduces to the erasure channel, so that C4 = 1 — d. In any 
other case, we could not find a closed-form expression of C4, and still resorted to the BAA. To 
run the BAA, we need the transition probabilities characterizing the channel, as those reported 
in Table |ni] for the case L = 2. We point out that, unlike the auxiliary channel considered in 
Section UIl the transition probabilities now depend on the value of d, so that the BAA must be 
run for each value of the deletion probability. 

The upper bounds C3 and C4 are compared in Fig. |7] for three different values of L — in 
both cases, L = 17 is the largest value for which we could run the BAA. We point out that the 
difference between the two bounds, yet C4 is rigorously tighter for each value of L, tends to 
vanish as L increases. This is due to the fact that, for large values of L, the number of deletions 
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Fig. 7. Different upper bounds on tiie capacity of tlie deletion channel. 



for every L transmitted bits is very likely to be close to dL, so that the advantage of knowing the 
actual number of such deletions (as it happens to the transmitter for the system with capacity C3) 
tends to vanish. As for the bound C3, for each value of L for which we could run the BAA, 
the bound C4 improves as L increases, and we conjecture that this behavior holds for any value 
of L (see Section I VIII) . 

VII. Discussions on the Proposed Upper Bounds 

In Table |IVl we report a comparison between the best upper bounds found in this paper, that 
is, C4 with L = 17 for (i < 0.83 and C| with Lmax = 17 for d > 0.83, and the existing upper 
bounds that we are aware of. We remark that the proposed approaches lead to a new state-of- 
the-art upper bound on the capacity of the deletion channel for most values of d, as evident 
from the table (where the best values are shown in bold face). 

We believe that the values reported in Table |IV] could be improved if it were possible to run 
the BAA for longer sequences. In particular, our conjecture is formalized in the following. 
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0.100 


0.100 
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0.95 


0.050 
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0.025 
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0.040 
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not given in [7] 


0.015 


0.98 


0.020 


not given in [7] 


0.010 


0.99 


0.010 


not given in [71 


0.005 



TABLE rV 



Different upper bounds on the capacity of the deletion channel. 



Conjecture 1: 

• the bound Ci does not worsen as D increases; 

• the bound C2 does not worsen as R increases; 

• the bound C3 does not worsen as L increases; 

• the bound C4 does not worsen as L increases. 

These conjectures are based on the amount of genie-aided information, that is, the entropy 
per input bit of the revealed processes. The idea is that the lower the entropy per input bit of the 
revealed information, the tighter the upper bound. For example, let us consider the bound Ci. if 
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we reveal the position of one deletion every 100, we expect a tighter bound than if we reveal the 
position of one deletion every 3. Unfortunately, we could not completely prove the conjectures 
listed above, but we were able to derive closely related results. For example, we can prove 
that C4 does not increase when L is replaced by any positive multiple of L. It is sufficient to 
note that, when L = ^,Y carries the same information as when L = ni (Vn > 0), plus some 
additional information. Hence, we get 



max /(Xj; Y, 



< n max /(Xj: Yjl 



which, according (|29l) . proves that C4 does not increase when L = £ is replaced by L = ni. 

We now discuss the behavior of the proposed upper bounds for limiting values of d, that is, 
(i — s> 0+ and d — > 1". In particular, after straightforward manipulations, the following results can 
be obtained 

1 — C* 

lim = a(R+l,R) + l = a(R + l,l) + l (30) 

d^0+ d 

1 — C 

lim = - 1) + 1 = a(L,l) + 1 (31) 

d^Q+ d 

lim-^ = i_ "(-^^^AX,i?) 
d^i- 1-d R+1 

which are valid for any finite value of R, L, and Lmax- The limits reported above are the only 

ones leading to closed-form expressions that do not reduce to the trivial erasure-channel bound. 

The limit for small values of d is determined by the coefficient a{L, 1), some values of which 

are reported in Table |V] — note that the coefficients in (|30l ) and (|3T| ) are identical, except for 

the name of the parameters. The best value that we have found so far is 

lim = 4.19 (33) 

d^0+ d 

obtained when L = 22. Other than the erasure-channel bound, we are not aware of any upper 
bound that leads to closed-form limiting expressions comparable with the reported one. We 
believe that (|33l) could be improved if it were possible to run the BAA for longer sequences, as 
formalized in the following. 
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L 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


a(L,l) 


2.08 


2.21 


2.33 


2.44 


2.55 


2.64 


2.73 


2.82 


2.90 


2.98 


3.05 


3.12 


3.19 



TABLE V 

Coefficient a(L, 1). 



Conjecture 2: For all values of L, the following holds 

if L> L then 1) > a(L, 1) . (34) 

We wish to prove this conjecture since it would imply that the asymptotic upper bound (|3TI) 
does not worsen as L increases. Additionally, a strict inequality in (|34l) . which holds for all 
available outcomes of the BAA, would imply that the asymptotic upper bound (|3TI) improves as 
L increases. Lemma 6 gives a partial proof of (l34l) . We point out that the limiting value (|3TI) 
may not be limited, since (flTI) does not satisfy any convergence criterion [11]. 

The limit for large values of d leads to similar considerations. In particular, the best value 
that we have found so far is 

lim — ^ = 0.49 , (35) 

d^i- 1 — a 

obtained by (l32l) when R = 8 and Lmax = 17. Note that, according to dH), the reported 
value could be improved by running the BAA for longer sequences, which unfortunately seems 

infeasible. We point out that (|35l) improves the limiting upper bound 

(J 

lim < 0.7918 

d~*i' 1 — a 

derived in [7], and closes the gap from the limiting lower bound 

C 

lim > 0.1185 

d^i- 1 — a 

derived [12]. 
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VIII. Two Simple Lower Bounds 

In this section, we derive lower bounds on C by exploiting the random process V defined in 
Section |Vl For any input distribution -P(X), the following equation holds 

/(X; Y) = /(X; Y, V) - /(X; V| Y) (36) 

by definition [2]. Moreover, since /(X; V|Y) cannot be larger than the entropy Hiy) of the 
process V, we can write 

/(X;Y)>/(X;Y,V)-if(V) (37) 

from which we get the following lower bound on the capacity of the deletion channel 

C> lim 47^(X;Y,V)- lim ^,Hi\) . (38) 

If we consider the process V defined before, following the arguments given for the derivation 
of (|29l ), we obtain 

so that (t38l) can be written as 

1 1 ^ 

C > -I{Xf, Yi) + yJ2p(L^ ^) log2 [P{L, R)] . (39) 

R=0 

In Fig. [8l the lower bound (l39l) is compared with the best lower bound available in the 
literature, namely the one from [4] or the one from [5], depending on the value of d (see 
Section U). For the computation of (|39| ). two different input distributions have been considered, 
that is, the distribution that maximizes /(X^; Yj), which was considered in the previous section 
to derive C4, and lUD input bits. Note that the difference between the curve related to the 
optimized input distribution and that related to lUD input bits is not significant for low values 
of d, which is compliant with the fact that lUD input bits are optimal when d = 0. Interestingly, 
for low values of d, both distributions lead to a lower bound roughly as good as the reference 
benchmarks, as evident from Table |VI] (where the best values are shown in bold face). 
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Bound from [3] 


Bound from [5] 


Bound J39t. L = 17, optimized input 


Bound ([39), L = 17, lUD input 


0.01 


0.919 


not given in [5] 


0.921 


0.921 


0.02 


0.858 


not given in [5] 


0.862 


0.862 


0.03 


0.805 


not given in [5] 


0.811 


0.811 


0.04 


0.757 


not given in [5] 


0.766 


0.765 


0.05 


0.713 


0.728 


0.724 


0.722 


0.10 


0.531 


0.562 


0.555 


0.546 



TABLE VI 

Different lower bounds on the capacity of the deletion channel. 



IX. Conclusions 

We have presented novel upper bounds on the capacity of the IID binary deletion channel. All 
bounds have been obtained by revealing side information on suitable random processes, and by 
computing the capacity of the resulting genie-aided systems. The proposed approaches lead to a 
new state-of-the-art upper bound for most values of the deletion probability d, and provide novel 
insights on the channel capacity in the limiting scenarios c? ^ 0+ and d — 1^. As a by-product 
of our approach, we have also presented simple lower bounds, which turn out not to improve 
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the existing ones. 
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